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ABSTRACT 

A brief overview of NASA’s recent experience in supercomputing is presented from 
two perspectives: early systems development and advanced supercomputing applications. 
NASA’s role in supercomputing systems development is illustrated by discussion of 
activities carried out by the Numerical Aerodynamic Simulation Program. Current capa- 
bilities in advanced technology applications are illustrated with examples in turbulence 
physics, aerodynamics, aerothermodynamics, chemistry, and structural mechanics. Capa- 
bilities in science applications are illustrated by examples in astrophysics and atmo- 
spheric modeling. The paper concludes with a brief comment on the future directions and 
NASA’s new High Performance Computing Program. 


1. Introduction 

The National Aeronautics and Space Administration (NASA) has for many years 
been one of the pioneers in the development of supercomputing systems and in the appli- 
cation of supercomputers to solve challenging problems in science and engineering. 
Today supercomputing is an essential tool within the Agency. There are supercomputer 
installations at every NASA research and development center and their use has enabled 
NASA to develop new aerospace technologies and to make new scientific discoveries that 
could not have been accomplished without them. 

NASA’s experience with advanced supercomputers began in the 1970s with the 
installation of the Illiac-IV parallel computer at Ames Research Center and the CDC 
STAR- 100 vector computer at Langley Research Center. These two systems were the 
first of a new generation of high-performance computers designed to achieve dramatic 
performance increases through the architectural innovations of parallel and vector pro- 
cessing. Although these systems were marginally successful, they provided a valuable 
testbed environment in which to learn to effectively use parallel and vector computing 
techniques. Architectural investigations continued into the 1980s with the development of 
the Massively Parallel Processor, a massively parallel SIMD processor system, at 
Goddard Space Flight Center, and the Cosmic Cube/Hypercube, parallel MIMD systems, 
at the California Institute of Technology and NASA’s Jet Propulsion Laboratory. In the 
1980s NASA also established the Numerical Aerodynamic Simulation (NAS) Program to 
pioneer supercomputing technologies and make them available to the aerospace com- 
munity. The NAS Program, for example, acquired the first full-scale Cray-2 and first 
Cray YMP computer systems and developed the first supercomputer UNIX environment. 



NASA’s involvement in supercomputing applications also began in the 1970s. 
Acquisition of the Illiac-IV, STAR- 100, and Cray- 1 computers spurred the development 
of innovative parallel computing algorithms. New computational disciplines, such as 
computational fluid dynamics and computational chemistry soon emerged and it became 
clear that supercomputers were a truly enabling technology. Today, supercomputers are 
being applied routinely to advance technologies in aeronautics, transatmospherics, space 
transportation, and space exploration, and to study the Earth, other planetary systems, and 
objects in outer space. 

The purpose of this paper is to provide a brief overview of selected NASA activities 
in supercomputing from two perspectives: supercomputing system development and 
supercomputing applications. The NAS Program is used as an example of NASA’s effort 
to remain at the leading edge of supercomputing system technology and bring this tech- 
nology to the aerospace research and development community. NASA’s use of super- 
computers in advanced technology applications is illustrated through examples in turbu- 
lence physics, aerodynamics, aerothermodynamics, chemistry, and structural mechanics. 
Examples in atmospheric science and astrophysics are presented to illustrate supercom- 
puting applications in science. Finally, the paper concludes with a look into future 
directions in high-performance computing. 


2. Numerical Aerodynamic Simulation Program 

The NAS Program 1 has two major goals. The first goal is to provide a national com- 
putational capability that is available to NASA, Department of Defense (DOD), other 
government agencies, industry, and universities, as a necessary element in ensuring con- 
tinued leadership in computational aerodynamics and related disciplines. The second goal 
is to act as a pathfinder in advanced large-scale computing capability through systematic 
incorporation of state-of-the-art improvements in computer hardware and software tech- 
nologies. The NAS Program began full operation in 1986 and today provides service to 
over 1700 users at 133 locations in the United States. 

The NAS complex of computers, called the NAS Processing System Network 
(NPSN), is shown in Fig. 1. It currently includes Cray-2 and Cray YMP/8128 supercom- 
puters, an Amdahl 5880 mainframe computer and associated mass storage system with 
2.3 terabytes of on-line capacity, 5 VAX minicomputers, 80 Silicon Graphics work- 
stations, a 32,000-processor Connection Machine parallel computer, and a 128-processor 
Intel Touchstone Gamma prototype parallel computer. All computer systems are linked 
through an extensive local area network combining Ethernet, HYPERchannel, and Ultra- 
net technologies to provide hardware data rates ranging from 10 to 800 mbits/sec. All 
computers operate under the UNIX operating system with DOD internet (TCP/IP) net- 
work communications provided via the well-known Berkeley UNIX “r” commands. 
Thus, a user can access any computer, run jobs, and transfer data among computers using 
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Fig. 1 NAS Processing System Network (NPSN). 


a single set of commands on all computers. A more complete description of the NPSN is 
given in references 2 and 3. 

Remote users access the NPSN through a wide choice of existing national communi- 
cation networks including the DOD-sponsored MILnet, the National Science Foundation- 
sponsored NSFnet, and NASA’s Science Internet. NAS is also a member of the Bay Area 
Regional Research Network that provides 1544 kbits/sec links between Ames Research 
Center, Bay Area universities, Department of Energy laboratories, and industry. MILnet 
is the main communication path to DOD laboratories, while NSFnet, and BARRnet serve 
the university community. For NASA and aerospace industry users, NAS has developed 
NASnet. NASnet uses Ethernet bridging technology to provide fast response and high 
throughput to remote graphics workstation users. NASnet currently serves 31 remote sites 
at bandwidths ranging from 56 to 1544 kbits/sec. 

The NAS Program’s experiences in advanced supercomputer technology began in 
1984 when it decided to acquire Cray Research, Inc.’s Cray-2 computer, which was in 
design at the time. NAS, in collaboration with the Department of Energy, began proto- 
type testing in 1985. In late 1985, NAS took delivery of the first full-scale Cray-2 config- 
uration. This system had a memory capacity of 2.048 gigabytes (over 268 million 64-bit 
words) which was 34 times the central memory capacity of the Cray XMP/48. The Cray- 
2’s very large memory, coupled with its ability to sustain a computing rate of 250 million 
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floating-point operations per second, allowed computational fluid dynamics researchers 
to increase their application program size by more than an order of a magnitude and 
increase their complexity without significant code modification. The very large memory 
virtually eliminated memory-disk transfer overhead during execution of large programs. 
The net effect was decreased preparation time for researchers tackling increasingly com- 
plex problems, plus improved job turnaround time. More importantly, researchers were 
able to perform fluid dynamics simulations about realistic three-dimensional (3-D) 
aerospace vehicle configurations that were not possible just a few years before. 

Providing a uniform user interface and operating environment across subsystems was 
a major NPSN design goal. Furthermore, the environment had to remain common across 
subsystems even as new supercomputer generations and other new subsystems were 
added. Only an open system architecture (implementation based on openly available defi- 
nitions for hardware and software interfaces) that was free of vendor proprietary restric- 
tions could meet this goal. UNIX operation system, DOD internet protocols, and the 
Berkeley network interface were chosen and implemented on the Cray-2. Prior to this, 
UNIX had only been implemented on minicomputers and workstations, and many felt 
that UNIX was not suitable for supercomputers. Today, UNIX is the standard for super- 
computer operating systems. The key to the successful implementation on the Cray-2 and 
later on other supercomputers was to treat UNIX as a definition that could be imple- 
mented and, if necessary, enhanced to meet supercomputing needs. For example, a batch 
resource management system, NQS, was added to provide users the flexibility of both 
interactive and batch supercomputing. Moreover, UNIX, when coupled with common 
networking commands, creates a so-called seamless network environment that enables 
users to move easily between systems, allows for easy file access and command initiation 
across machine boundaries, and enhances user-code portability. Finally, the open archi- 
tecture concept allows for modularity essential for ready implementation of new capa- 
bilities and has proved beneficial in the installation of the Cray YMP and other new 
systems. 

Enhanced user productivity was another major NPSN design goal. To meet this goal, 
the NPSN was designed to include scientific workstations as the researcher’s interface to 
supercomputers. The design was implemented by installing NAS Silicon Graphics IRIS 
workstations and connecting them directly to the Cray-2 though a local area network. 
This was a unique implementation at the time and has since been replicated at many 
supercomputer installations. Combined with UNIX and network software, the 
supercomputer becomes literally an extension of the workstation. The user resides at his 
workstation where text and small data files are created and modified, where applications 
are submitted and interacted with, and where complex input data and results are displayed 
and analyzed. Because numerical simulations produce an enormous amount of data, the 
workstation’s graphics display capability is especially beneficial. By using graphics 
workstations, researchers can digest results in a reasonable amount of time and gain a 
qualitative picture of the physical phenomenon simulated. Numerous graphics packages 
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have been developed to operate across workstations and Cray supercomputers in a 
distributed manner and to allow researchers to display any facet of their data for easy 
interpretation and evaluation 4 ' 7 . Today, interactive graphical post-data processing is an 
essential aspect of today’s sophisticated numerical simulation process. 

Providing ready access by the national aerospace research community from their 
home locations was another NPSN design goal. This led to the development of NASnet, a 
unique high-performance nationwide communication network that links remote user 
locations to the NAS facility via land-links, with communication speeds up to 
1544 kbits/sec. NASnet connects the communication lines from the NAS facility to local 
area networks at each remote site through Ethernet bridges and thereby creates an exten- 
sion of the NPSN local network. The implementation of UNIX and network software on 
remote workstations provides an environment in which remote researchers have virtually 
the same interactive capability as users at the NAS facility itself. Presently, NASnet is 
being redesigned to utilize new internet gateway technology and thereby increase its 
performance to approximately 6200 Kbits/sec. 

3. Computational Fluid Dynamics 

Computational fluid dynamics has been one of the earliest drivers in advancing 
supercomputing within NASA. The introduction of numerical techniques in the late 
1960s to treat highly nonlinear flows, such as transonic flow over lifting airfoils with 
embedded shock waves 8 , rapidly led to greatly increased interest in computational solu- 
tions to fluid flow problems. The discipline grew rapidly with improvements in both 
computer systems and algorithms. Fig. 2 shows that improvements in computer perfor- 
mance have been closely paralleled by improvements in numerical methods 9 . The data 
shown here illustrates that the cost of performing a given calculation has decreased three 
orders of magnitude due to advances in computers and also three orders of magnitude due 
to improvements in numerical methods over a 15-year period. While the example shown 
is for methods used to solve two approximating forms of the Navier-Stokes equations that 
govern the fluid dynamics of a perfect gas, it is indicative of the trend in other disciplines 
as well. 

Computational fluid dynamics is now a well-established discipline within NASA and 
the aerospace industry 10 - 11 . While industry’s main focus is on design applications, NASA 
is more involved in research. The objectives of NASA’s efforts, which are discussed 
below, are to (a) advance fundamental understanding of basic fluid dynamic processes 
and (b) develop an advanced base of new aerodynamics technology for application to 
future generations of aerospace vehicles and propulsion systems. 
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Fig. 2 Comparison of numerical simulation cost trend resulting from improvements in fluid dynamics 

algorithms and in computers. 


3.1 Turbulence Physics 

Turbulence is the most common fluid motion encountered in nature. It greatly affects 
the resistance of bodies moving through fluids, fluid mixing, and heat transfer. Because 
of its importance, turbulence study has attracted some of the best scientific minds for over 
a century. Despite this attention, turbulence is still not well enough understood to be 
effectively controlled in engineering applications. If it could be controlled, the benefits 
would be significant. For example, in the case of a fleet of transport aircraft a 20 percent 
reduction in fuselage skin friction alone would save approximately $1 billion a year in 
fuel consumption 12 . 

With the development of supercomputers and highly accurate numerical algorithms, it 
is possible to obtain numerical solutions to the Navier-Stokes equations for simplified 
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geometry and at low Reynolds numbers. These numerical simulations are based on first 
principles and can be treated as though they were observed data from a physical experi- 
ment. Used in this way, numerical simulations are providing new insights into interrela- 
tionships between flow variables, such as the relationship between pressure and strain for 
which no measurement has yet been possible. 

In addition to quantitative information of unprecedented detail, numerical solution 
offers, for the first time, the opportunity to visualize turbulent flows as they evolve in 
both space and time without the averaging and filtering inherent in results obtained from 
laboratory experiments. Each structure in a turbulent flow can be inspected prior to calcu- 
lating the statistical characteristics of significant flow features. This visualization capabil- 
ity is valuable both for identifying new relationships among the various elements of the 
dominant flow structures and for suggesting new types of physical experiments. Fig. 3 
shows the calculated vortical structures near a flat plate 13 . These results were obtained 



Fig. 3 Calculated vortical flow structures in a turbulent boundary layer. 
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from the database provided by direct (first principles) numerical simulation of a flat-plate 
turbulent boundary layer 14 . The direct simulation provided pressure and all three compo- 
nents of the flow velocity for each time step and for each of 9.4 million grid points in the 
computational domain. A problem of this magnitude requires about 200 hours of CPU 
time on a Cray-2 computer. Turbulence physicists have analyzed the results shown in Fig. 
3 and have concluded that locations of significant contributions to Reynolds stresses 
occur adjacent to large hook-shaped vortical structures that are common in turbulent 
flows l5 . Information of this sort aids in the development of statistical models that 
accurately reflect the underlying physical behavior of turbulence. 


3.2 Aerodynamics 

Computational aerodynamics is concerned with numerical simulations of aircraft 
flying at normal altitudes and at Mach numbers below five where air behaves as a perfect 
gas. The computational models employed differ in levels of approximation to the govern- 
ing Navier-Stokes equations and in geometric complexity. Due to computer limitations, 
one generally is forced to trade geometric simplicity for less approximate solutions or 
more approximate solutions for geometric complexity 9 . As the approximations become 
less severe and the solutions become more complex, the computer resources needed for a 
solution grow rapidly 15 . Generally, approximations that neglect viscosity (inviscid) in the 
flow field and treat viscosity only in the boundary layer are in widespread use for aerody- 
namically clean configurations at or near cruise conditions. Formulations that include 
viscous terms become important when treating flows where the boundary layer is sepa- 
rated from the surface. These occur, for example, near boundaries of the flight envelope, 
in engine inlets and with those flows associated with vortex-enhanced lift. 

The level of modeling receiving the most attention within the NASA research envi- 
ronment is Reynolds-Averaged Navier-Stokes (RANS). RANS models include all terms 
of the Navier-Stokes equations averaged over a scale that is large with respect to the tur- 
bulent eddy fluctuations, yet small relative to the macroscopic flow changes. This formu- 
lation introduces terms representing the transport of energy and momentum between the 
mean and fluctuating component. These cannot be determined analytically and must be 
phenomenologically modeled. Turbulence modeling is the major item pacing the full 
exploitation of the RANS approach and continues to be a focus of experimental and 
computational research. 

Increases in computer performance have dramatically affected the utility of the RANS 
formulation 16 . Fig. 4 illustrates the effect of increasing supercomputer performance on 
advancing RANS simulation capability during the last 15 years. In 1975, a two- 
dimensional (2-D) calculation of transonic flow over a simple airfoil strained the capabil- 
ity of the CDC 7600 systems available at that time. The increased performance of the 
Cray-1 made it possible to compute simple 3-D wing flows by 1983. By 1986 it was 
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Fig. 4 Advances in Reynolds-Averaged Navier-Stokes simulation capability with increases in supercom- 
puter performance. 


possible to compute flows over wing bodies on the Cray XMP/4. Today the Cray YMP/8 
enables simulation about very complex geometries such as the space shuttle configuration 
shown at the bottom right-hand side. 

Fig. 5 shows more detailed results from the use of the RANS approximation to com- 
pute the transonic flow about the integrated space shuttle in ascent mode 17 . Here, com- 
puted surface pressures are compared with those obtained from wind tunnel tests. The 
computed and measured results are in excellent agreement over large portions of this 
complex configuration. A limited amount of flight test pressure data is available and is 
compared with wind tunnel and computed pressure along the side of the fuselage in 
Fig. 6. At this particular Mach number of 1.05, the computation is in better agreement 
with the flight data than with the wind tunnel data, apparently due to wind tunnel wall 
interference. Although there is disagreement found in localized regions, the computa- 
tional approach is considered adequate to predict the aerodynamic loads needed to refine 
the current flight envelope and to evaluate proposed emergency abort maneuvers. 


ORIGINAL PAGE IS 
OF POOR QUALITY 


9 




Fig. 5 Comparison of pressure contours between computation (outermost) and wind tunnel (innermost), 
Moo = 1.05, a = -3°, and Re = 4.0 x 10 6 /ft (3% model), (a) Side view, (b) top view. 


4 . Aerothermodynamics 

Aerothermodynamics combines the discipline of computational fluid dynamics with 
the modeling of real gas properties. Vehicles flying at Mach numbers greater than 
approximately five create shock waves strong enough to cause the chemistry of the air to 
change significantly. Therefore, to adequately simulate such flows, additional equations 
governing the behavior of the real gas chemistry must be added to the Navier-Stokes 
equations. Clearly, the level of computational complexity increases over that of computa- 
tional fluid dynamics alone. 
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Fig. 6 Comparison of pressure coefficient (CP) from computation (-), wind tunnel (o), and flight test 
(V right side, A left side) along <|> = 70° line side of the orbiter fuselage, Moo = 1.05, a = -3°, Re = 4.0 x 
lOfyft and 10°/9° elevon deflection. 

The goal of computational aerothermodynamics is to predict aerothermal environ- 
ments both in the case of external flow around vehicles and the internal flow within 
propulsion units. The physical phenomena which must be addressed include aerodynamic 
forces and moments, convective and radiative heating rates, gas and surface interactions, 
catalytic surfaces, interactions with active cooling of Thermal Protection System (TPS) 
materials, and plasma layers and their effect on electronic communications to and from 
the vehicle. Understanding these phenomena is critical to the design of the next 
generation aerospace transportation systems and to ensure that they are adequately pro- 
tected from the very high thermal loads encountered in hypervelocity flight. At the 
current state-of-the-art, the RANS equations coupled with the real-gas effects form the 
basis for the solutions to these phenomena. 

Real-gas effects include radiative thermochemical nonequilibrium where finite-rate 
processes for chemical- and energy-exchange phenomena occur. To account for reactions, 
conservation equations for each chemical species must be added to the RANS equation 
set. For a relatively simple case of dissociating and ionizing air, one must typically 
consider nine air species. Including the associated conservation equations for each species 
nearly triples the number of equations to be solved. In addition, to account for thermal 
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nonequilibrium there are additional energy equations to describe the energy exchange 
between the various energy modes (translational, rotational, vibrational, and electronic). 
Finally, when thermal radiation is important, the radiative flux must be included in the 
energy equation. 

Supercomputers have been essential in the development of computational aerother- 
modynamics and have permitted the modeling of complex 3-D flow fields with finite-rate 
chemistry. As an example, Fig. 7 shows the thermochemical nonequilibrium flow field 
about the Aeroassist Flight Experiment (AFE) vehicle at one of its trajectory points 18 . 
The left side of the figure shows computed normalized density contours. Superimposed 
on these are circles representing the location of the bow shock as determined from 
experimental shadowgraphs obtained from ballistic range tests. The experimental and 
computational bow shock locations are nearly identical. Shown on the right side of the 
figure is a comparison of the vibrational and translational stagnation line temperatures. 
As illustrated, there is a significant difference in the two temperatures. This difference, 
depending on the entry conditions, can result in a significant increase in the total radiative 
heating to the vehicle 19 . 




Fig. 7 Thermochemical nonequilibrium flow field about the aeroassist flight experiment (AFE) vehicle, 
(a) Computed density contours, computed and experimental bow shock locations; (b) vibrational and 
translational temperatures along the stagnation line. 
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5. Computational Chemistry 

During the past decade, computational chemistry has become a dynamic and versatile 
tool to assist in understanding the fundamental nature of a variety of substances, notably 
gases, materials, and their interactions. For example, computational chemistry methods 
are currently being used to accurately predict radiative intensity factors, high-temperature 
transport properties, reaction rates, and rates of energy exchange during particle colli- 
sions. These data are required by computational aerothermodynamics to simulate both 
external and internal flow fields of aerospace vehicles. In addition, computational chem- 
istry methods are being used to study and design space-based materials which are more 
resistant to the harsh environment of space and, in particular, the impact of energetic 
oxygen atoms. 

Computational chemistry involves solving the Schrodinger equation to determine the 
properties of atoms, molecules, and matter. The techniques used to solve this equation are 
well-documented and will not be described here. It should be noted, however, that super- 
computers have had a profound influence on computational chemistry. While the perfor- 
mance gains of an order of magnitude have naturally allowed much larger problems to be 
considered, supercomputers have had a far greater impact on computational techniques. 
For example, the ability to perform Full Configuration Interaction (FCI) calculations in 
realistic basis sets has provided researchers with detailed benchmarks for evaluation of 
more approximate methods. These FCI calculations 2 ®, using the Cray-2 and Cray YMP, 
have produced results that are superior in accuracy to those determined from experimen- 
tal techniques. Furthermore, these results have demonstrated that multireference Cl wave 
functions reproduce the FCI results to very high accuracy and use approximately one- 
fourth of the computer time. Finally, the speed of supercomputers has tremendously 
reduced the time required to obtain results. Previously, even if a desired accuracy could 
be obtained, the real time required to obtain results was frequently too long for the results 
to be useful or to impact a specific program. 

High-temperature reaction rate constants for air and air plus hydrogen species are 
crucial to the accurate prediction of the external flow around hypersonic vehicles and 
modeling of the engine performance. Experimental rate constants are seldom available 
and almost impossible to obtain at high temperature. When data are available, they are 
often uncertain by several orders of magnitude. Fig. 8 is a comparison of the theoretical 
rate constant 21 for the three-body recombination of H + H + H2 with a variety of experi- 
mental data. As shown, at high temperatures the scatter in the experimental data is three 
orders of magnitude. Reaction rate constant calculations like these are providing data 
which do not exist elsewhere, as well as helping to clarify what is often a confusing maze 
of experimental data. 
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Fig. 8 Comparison of calculated and experimental rate constant for the three-body recombination 

ofH+H+H 2 . 


6. Structural Mechanics 

Computational structural mechanics is another major computational thrust within 
NASA and has been greatly influenced by supercomputers 22 . The finite element method, 
developed by the aerospace industry, is the predominant technique used for the analysis 
of solid and structural mechanics problems. In the early 1970s, NASA became heavily 
involved in finite element modeling through the development of the widely used 
NASTRAN finite element program. Today, finite element techniques employing super- 
computers are widely used to solve linear and nonlinear problems for both static and 
dynamic analysis. Application areas include structural analysis of aerospace vehicles, 
propulsion systems, and space structures. The application of advanced composite materi- 
als to vehicle and propulsion system structures, in particular, has required the develop- 
ment of much more sophisticated numerical models that have in turn placed increased 
demands on computer capability. 

An example of the application of supercomputers to a structural problem of critical 
importance is the space shuttle solid rocket booster (SRB) structural design assessment 
and redesign efforts following the accident that destroyed the Challenger. The accident 
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was believed to have been caused by the failure of a case joint in the right rocket motor ZJ . 
An analysis of the joint was performed at three levels of detail^ shown in Fig. 9. The 
first level consisted of a 2-D stress analysis for a shell model of the entire SRB that 
included an equivalent stiffness model of the joint. At the second level, the displacements 
calculated at level 1 were used as boundary conditions for a more refined 2-D shell model 
of an SRB segment including the joint. Finally, at the third level the displacement calcu- 
lated at level 2 was used as a boundary condition for a detailed 3-D analysis of a 1 -degree 
circumferential slice of the joint. Analysis was performed on both the original and 
redesigned joints. The results showed that the redesigned joint substantially reduced the 
gap motion at the O-ring surface and the sensitivity to O-ring performance. 

The SRB analysis was performed using various NASA computer systems. The 
researchers were located at Langley Research Center and used a minicomputer for model 
preparation and verification and for post processing. The 2-D shell finite element struc- 
tural analysis, using 54,870 equations, and the 3-D solid finite element analysis were per- 
formed on Cray supercomputers at Ames Research Center. This effort, performed in 1985 
and 1986, was a graphic illustration of how effectively supercomputers can be used from 
far remote locations. The solution time for the full SRB model (level 1) exceeded 
14 hours on a VAX 1 1/780 computer and was reduced to 14 minutes on a single Cray-2 



Fig. 9 Three levels of modeling detail for solid rocket booster (SRB) case joint structural analysis. 
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processor when fully vectorized. More recent algorithm improvements have reduced the 
solution time to 13 seconds on the Cray-2 using 4 processors and microtasking and to 
6 seconds on the Cray YMP using 8 processors and microtasking. In the latter case the 
9.2 billion floating-point operations for matrix factorization took 6 seconds, a rate of 
1.517 billion calculations per second. 


7. Astrophysics 

Supercomputers are also heavily used within NASA to perform astrophysical investi- 
gations. For example, the use of computational methods to investigate the formation and 
evolution of astronomical objects (stars, galaxies, radio sources, black holes. X-ray 
sources, interstellar clouds, etc.) has proven to be an essential part of the overall scientific 
approach to understanding the universe. Computation has affected nearly every area of 
astrophysics. It is a tool for interpreting observations that complements analytic theory 
and permits the study of complex phenomena that cannot be done in the laboratory. 

Applications of supercomputers to astronomy is too broad a topic to cover here; thus, 
attention is focused on computer application of galactic formation and evolution 25 . Three 
features drive present research efforts and give rise to the need for the computational 
power of supercomputers. These are (a) the complex play of many physical processes 
requiring nonlinear interaction models, (b) greatly improved observational detail at all 
wavelengths requiring complex source modeling, and (c) reduction of raw observational 
data requiring high-speed reduction, enhancement, and analysis. 

The study of the global dynamics of galaxies leads one to concentrate on the 
dynamics of the stellar population. The stars make up the bulk of the mass in most 
galaxies of interest. Stars move freely through these systems and they practically never 
collide. The collision mean free path is large compared to the system dimensions. The 
study of the stellar dynamics is then the study of a collisionless, gravitational n-body 
problem. The n-bodies are the stars, and they act like mass-points, or particles, that inter- 
act with the 1/r 2 forces of Newtonian gravitation. The models used for numerical 
experiments describe the time development of the self-gravitational particle system. Early 
experiments dealt with 2-D systems, but, with the advent of supercomputers in the late 
1970s, fully 3-D experiments were made possible. The number of particles followed in an 
experiment is typically 100,000. The forces necessary to integrate the equations of 
motion of the particles are obtained from potentials that are calculated on a grid. These 
potentials are calculated from the tabulated star densities by solving the Poisson equation 
at each integration step and the equations of motion are integrated by the time-centered 
leapfrog method. 

The approach described above has been used to study galaxy formulation and internal 
dynamics and the interaction of galaxies 2 ^. One example is the investigation of star for- 
mation stimulated by the collision of galaxies. Far infrared measurements from the 
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Infared Astronomy Satellite have revealed a class of very luminous interacting galaxies 
which appear to be starburst galaxies. These results have stimulated research on the 
nature of the interactions and the ways in which the star-formation process can be stimu- 
lated through the interaction. An example of an optical photograph of an interacting pair 
of galaxies is shown in Fig. 10. The photograph was made in such a way to emphasize the 
regions of high star formation. The observations show vividly the violent disruption of 
the two galaxies and the regions of high current star formation, particularly at the centers 
of the two galaxies. 

One of the computational approaches to understanding the starburst phenomenon lies 
in understanding the dynamical interaction of two spiral galaxies embedded in massive 
halos by numerical experiments. For example, one can follow the dynamical interaction 
of two disk galaxies to understand the nature of the contraction and subsequent expansion 
of the two galaxies. Fig. 1 1 shows four snapshots of a collision sequence in a numerical 



Fig. 10 Interacting galaxies UGC 12914/12915. (Photograph courtesy of Dr. H. Bushouse, NASA Ames 

Research Center.) 
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Fig. 1 1 Time development of numerical experiment on interacting galaxies as viewed from above the 

orbital plane. 


experiment 26 . The disks were oriented in the plane of the orbit and both the disk particles 
and the surrounding halo particles are shown. The particles shown are a sampling (2048) 
of the total number of particles in the calculation (100,000). The counter in the bottom 
right of the frames is the elapsed time in nondimensional units. The time sequence shown 
corresponds to roughly 150 million years. Understanding the nature of the interaction and 
the time variations of the densities and momenta in the two galaxies is the first step in a 
longer term project toward understanding the nature of the star-formation process in these 
systems. 


8. Atmospheric Modeling 

Atmospheric modeling on supercomputers has become a principal tool in predicting 
weather and studying climate change. The importance of this activity is clearly evident in 
light of the need to assess the influence of man’s activity on the environment. NASA has 
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been a major contributor to advancing the applications of supercomputers to this field 
including: global modeling of the interactions between the oceans, atmosphere, land and 
ice; simulation of the dynamics and chemistry of the Earth’s atmosphere including the 
response of the Earth’s ozone layer to changing chemical atmospheric composition; and 
assimilation of data acquired from spacebome sensing instruments. NASA is also 
engaged in studying the atmospheres of the planets. The study not only increases our 
understanding of the planets, but also lets us gain insight into the dynamics of the Earth’s 
atmosphere. 

The modeling of Mars’ water cycle is one example of the use of supercomputers to 
study planetary atmospheres. Spacecraft missions to Mars during the 1960s and ’70s have 
shown that the total amount of water vapor in its atmosphere fluctuates during the course 
of a year by an amount equivalent to about 1 km 3 of ice. This indicates that Mars has an 
active hydrological cycle in which water is exchanged between the atmosphere and vari- 
ous other reservoirs. The nature and distribution of these other reservoirs is not entirely 
clear, but it is known that one of them is the “residual” north polar cap — not the one that 
waxes and wanes with the seasons because that one is made of CO 2 frost — but the one 
that remains throughout the summer. In other words, when the seasonal CO 2 ice cap 
finally sublimes away in late spring, it leaves behind an underlying water ice cap that is 
less volatile and can survive the summer’s heat. This remnant polar cap serves as a source 
for atmospheric water vapor. Surprisingly, however, the seasonal CO 2 cap at the south 
pole never disappears. For reasons not understood, carbon dioxide survives at the south 
pole all year long. 

The significance of this asymmetry in polar cap behavior is that: if the north cap 
supplies water to the atmosphere each summer, and the south cap does not, then where 
does the water go? There have been many suggestions, but all of them are based on 
models that treat atmospheric transport so crudely that the results cannot be considered 
proof. It is this aspect of the problem, atmospheric transport, that requires supercom- 
puters. To model atmospheric transport properly, one needs to solve the full 3-D equa- 
tions of motion on a globe, and carry out the calculations for very long periods of time. 
The temporal aspect of the problem is important in order to know net annual gains or 
losses. Thus, multi-year simulations are required (one Mars year is about two Earth 
years). Even this, however, may not be enough to properly model the problem. The rea- 
son is that in addition to transport, there are important physical processes that occur, such 
as cloud formation, precipitation, boundary layer mixing, etc., that can affect the behavior 
of atmospheric water just as much as transport. Adding “physics” to the problem could 
easily increase the computational demand by an order of magnitude. 

During the past several years, NASA researchers at Ames Research Center have been 
using the Cray supercomputers at NAS to attack these problems. They have constructed a 
transport model as well as various physical models. The speed and memory of these 
supercomputers have enabled the examination of polar mixing processes, cloud 
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formation, and boundary layer exchange at a level of detail not previously possible 27 . For 
example, the ability of its atmosphere to mix water vapor into the polar regions during 
winter is a key issue concerning the current Mars climate system. In Fig. 12 contours of a 
nearly conserved dynamical quantity (isentropic potential vorticity, IPV) are shown to 
illustrate this mixing process. The IPV field is taken from a simulation of the Martian 
winter circulation using a 3-D climate model being developed by researchers at Ames 
Research Center and based on the work in reference 27. The center of the figure is the 
north pole while the edges are at about 30 degrees north latitude. The large gradients in 



Fig. 12 Simulated isentropic potential vorticity (IPV) field at the Martian north pole in winter. 
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the IPV field just poleward of the dark ribbon surrounding the pole represent a barrier 
that large-scale wave motions must break down and penetrate if mixing is to occur. At 
this particular time a wave number of three disturbance is evident, but its amplitude is 
small and the mixing it produces is weak. For this particular experiment, therefore, the 
Martian polar vortex is relatively stable and similar in this respect to the Earth’s Antarctic 
stratosphere during winter. By running the transport code at high temporal and spatial 
resolution, for example, it was found that the wintertime circulation on Mars was sluggish 
in its ability to move water to the very high latitudes, which is contrary to earlier sugges- 
tions. Similarly, we have found that when clouds do form on Mars, they do not readily 
precipitate water to the ground; this also is contrary to earlier suggestions. 

Recently supercomputers have been used to solve one of the most perplexing prob- 
lems of atmosphere-surface interactions on Mars 28 . Namely, how does water move back 
into the surface? All previous studies indicate that once water is driven out of the soil, it 
does not readily return. By running a coupled boundary layer/soil model at very high ver- 
tical and temporal resolution, a physical process has been discovered that facilitates the 
return. This will significantly affect the field and open up new areas of research. Similar 
statements can be made about recent transport and cloud studies. 


9. A Look to the Future 

To meet future challenges in computational science, NASA has initiated the High 
Performance Computer (HPC) Program. The goal of the HPC Program is to accelerate the 
development and application of high-performance computing technologies to meet 
science and engineering requirements for continued leadership in aerospace. Timely real- 
ization of the goals in many of NASA’s science and engineering missions require signifi- 
cantly increased computing power. For example, in the field of computational fluid 
dynamics, the governing equations have been known for over a century, but, it has 
required supercomputer-level performance to develop tools for understanding the physics 
of aerodynamic flows and for aerospace vehicle and propulsion system design. 

Recently, multidisciplinary methods have appeared. These methods are used to 
perform computer simulations of coupled phenomena such as those needed to study wing 
flutter. Flutter is the motion due to the complex interaction of the instantaneous changes 
in wing shape due to aerodynamic forces and the instantaneous change in aerodynamic 
forces due to wing shape. The effect of flutter can be catastrophic if dynamic forces 
exceed structural limits. However, flutter can, in some cases, be suppressed by a coupled 
deflection of control surfaces. Simulation of flutter control requires combining fluid 
dynamics, structural mechanics, and control laws into a time-dependent formulation. One 
example of simulation of wing flutter control is presented in Fig. 13. In this work 2 ^, the 
results were obtained by solving the model structural equations of motion, a simple con- 
trol law equation, and a nonlinear inviscid form of the fluid dynamic equations (full 
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AEROELASTICALLY OSCILLATING WING 



CONTROL SURFACE ACTIVATED 



ACTIVE CONTROL SUPPRESSES OSCILLATION 


Fig. 13 Computed pressures on a low aspect ratio wing with and without a control surface activated to 
suppress flutter. Moo = 0.9, a = 0, simulated altitude = 30,000 ft. 
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potential) in a fully coupled manner. Wing flutter is established with the control surface 
fixed (upper figure). Next, the control surface is activated (middle) so that its instanta- 
neous deflections reduce and maintain the wing steady (bottom). The various shades of 
gray on the wing surface represent the levels of instantaneous pressure on the wing. 

The example above was a highly approximate treatment for a simple shape. Today’s 
aeroscience challenge is to provide integrated, multidisciplinary simulations of aerospace 
vehicles throughout their mission profiles. This requires the simulation of propulsion 
systems, complete external aerodynamics, structural response, controls, and chemistry in 
an integrated manner. It is estimated that these calculations will take tens of hours on 
computer systems capable of sustaining a trillion floating-point operations per second 
(teraFLOPS). 

In space and Earth sciences, NASA has made significant strides in weather/climate 
simulation, simulation of stellar/galactic evolution, solid Earth modeling, and analysis of 
multi-spectral/multi-sensor data. Today’s space and Earth sciences challenges are the 
multidisciplinary modeling and monitoring of the Earth and its global changes, assess- 
ments of their potential impact on the future environment of man, better understanding of 
space phenomena, formation of the universe, and the origins of life. To begin addressing 
the complex interdisciplinary modeling for such nonlinear problems will require 
teraFLOPS of computational power and terabytes of memory capability. 

In many cases the computational power to solve these problems can be located in 
ground-based facilities, but in others, such as multi-sensor airborne or satellite systems, 
the computational power must be located in a flight environment. Highly autonomous 
space exploration systems, e.g.. Mars Rover sample return, will require high-performance 
and ultra reliable spacebome computers. These specific grand challenges and others 
facing the Agency provide a computational imperative for a NASA initiative in High 
Performance Computing. 

The approach to be taken by the HPC Program is to fully exploit highly parallel com- 
puting technology by focusing on system architecture, highly parallel algorithms and 
basic computer science research. One of the program’s specific objectives is to develop 
technologies that will enable a sustained computing rate of a teraFLOPS; approximately 
1000 times the sustained performance of an 8-processor Cray YMP. Highly parallel pro- 
cessor technology was chosen because it presents the best alternative to the conventional 
supercomputer approach. The conventional approach depends on a small number of very 
fast processors sharing a very fast common memory. This approach will be severely lim- 
ited in the future by electronic technology limits (speed of light, circuit density, and 
electronic packaging), processor to memory bandwidth, and high development costs. 
Highly parallel processors greatly increase parallelism, relax demands on technology, and 
share development costs with the much larger microcomputer market. Parallel processing 
research has been ongoing within the NAS program and within other NASA programs for 
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the past few years. The primary thrust has been to evaluate parallel processing architec- 
tures and to develop efficient parallel applications. Results have been encouraging. For 
example, results have been obtained that show a 16,000-processor CM-2 will out-perform 
a Cray-2 single processor for certain large computational fluid dynamics applications 30 . 

Developing application algorithms and architecture capable of fully utilizing highly 
parallel concepts is another objective of the program. High-performance, parallel proces- 
sor testbed facilities will be implemented for developing both applications and systems 
software. They will be scalable so that initial feasibility can be demonstrated at smaller 
scales and at reduced cost. The applicability of new parallel processor architectures and 
algorithms will be demonstrated on selected multidisciplinary research applications. 
Three areas have been chosen: 

(a) Computational Aerosciences: integrated, multidisciplinary simulation of 
aerospace vehicles throughout their mission profiles 

(b) Earth and Space Sciences: multidisciplinary modeling and monitoring of the 
Earth and its global changes and assessment of their impact on the future 
environment 

(c) Remote Exploration and Experimentation: extended-duration human exploration 
missions and remote exploration and experimentation 

Finally, the basic computer science research infrastructure within NASA will be 
enhanced both at NASA research centers and at several NASA computationally oriented 
institutes. Strengthening basic research is crucial to achieving the innovative approaches 
needed to meet the initiative’s goals and exploit newly obtained capabilities. 

In summary, highly parallel computers offer potential to leap-frog the current state-of- 
the-art and accelerate the growth in computer speed. NASA has unique needs in aeronau- 
tics and in Earth and space sciences for greatly increased speed. HPCI is aimed at meet- 
ing these needs by providing the technology that before the end of the decade can achieve 
a teraFLOPS or a 1000-fold increase over today’s capability. This achievement will 
enable performance optimization, time and cost reduction, and increased safety and reli- 
ability in aerospace vehicles, and establish a powerful resource to maximize scientific 
benefits of space observations. 
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